Determining visual tracking points for frame-based cameras are known. Similarly, determining visual tracking points for event-based cameras are also known. However, the visual tracking points from frame-based cameras compared to event-based cameras are typically different points due to the nature of the camera types. Systems and methods for determining improved tracking points are needed.
In one embodiment, an electronic device provides for synchronizing frame-based and event-based tracking points. The electronic device includes a frame-based camera, an event-based camera, and a processor. The processor is coupled to the frame-based camera and the event-based camera. The processor receives frame-based data captured by the frame-based camera and event-based data captured by the event-based camera; identifies a frame-based tracking point from the frame-based data; identifies an event-based tracking point from the event-based data corresponding to the frame-based tracking point; and synchronize the frame-based camera and the event-based camera based on the identified frame-based tracking point and the identified event-based tracking point.
In a second embodiment, a method provides for synchronizing frame-based and event-based tracking points. The method includes receiving frame-based data captured by a frame-based camera and event-based data captured by an event-based camera; identifying a frame-based tracking point from the frame-based data; identifying an event-based tracking point from the event-based data corresponding to the frame-based tracking point; and synchronizing the frame-based camera and the event-based camera based on the identified frame-based tracking point and the identified event-based tracking point.
In a third embodiment, a non-transitory medium embodying a computer program provides for synchronizing frame-based and event-based tracking points. The program code, when executed by at least one processor, causes a processor to receive frame-based data captured by a frame-based camera and event-based data captured by an event-based camera; identify a frame-based tracking point from the frame-based data; identify an event-based tracking point from the event-based data corresponding to the frame-based tracking point; and synchronize the frame-based camera and the event-based camera based on the identified frame-based tracking point and the identified event-based tracking point.
In a fourth embodiment, a head mounted display (HMD) provides for synchronizing frame-based and event-based tracking points. The HMD includes an inertial motion unit (IMU), at least one optical sensor, and a processor. The IMU generates motion data based on an HMD movement. The processor is coupled to the IMU and the at least one optical sensor. The processor executes instructions to receive data from the IMU; receive low resolution data and high resolution data at different framerates from the at least one optical sensor; check whether data is available from a more precise source for each data point received; update an offset for a less precise data source based on the more precise source when available; and determine pose based on the updated offset.
In a fifth embodiment, an HMD provides for synchronizing frame-based and event-based tracking points. The HMD includes an inertial motion unit (IMU), at least one outward facing optical sensor, and a processor. The IMU is configured to generate motion data based on an HMD movement. The at least one outward facing optical sensor is coupled to the HMD. The processor is coupled to the IMU and the at least one optical sensor. The processor executes instructions to receive motion data from the IMU; determine rate of pose change of the HMD based on the received motion data; and adjust an optical tracking rate from the outward facing optical sensor based on the determined rate of pose change.
In a sixth embodiment, an HMD provides for synchronizing frame-based and event-based tracking points. The HMD includes an outward facing frame-based camera, an outward facing event-based camera, an inertial motion unit (IMU) and a processor. The frame-based camera is coupled to the HMD. The event-based camera is coupled to the HMD in proximity to the frame-based camera. The IMU generates motion data based on an HMD movement. The processor is coupled to the frame-based camera, the event-based camera, and the IMU. The processor executes instructions to receive motion data from the IMU; select at least one of the frame-based camera or the event-based camera based on the motion data; receive image data from the selected camera; and process received image data to determine an HMD orientation.
Other technical features may be readily apparent to one skilled in the art from the following figures, descriptions, and claims.